27 research outputs found
Classification and Geometry of General Perceptual Manifolds
Perceptual manifolds arise when a neural population responds to an ensemble
of sensory signals associated with different physical features (e.g.,
orientation, pose, scale, location, and intensity) of the same perceptual
object. Object recognition and discrimination requires classifying the
manifolds in a manner that is insensitive to variability within a manifold. How
neuronal systems give rise to invariant object classification and recognition
is a fundamental problem in brain theory as well as in machine learning. Here
we study the ability of a readout network to classify objects from their
perceptual manifold representations. We develop a statistical mechanical theory
for the linear classification of manifolds with arbitrary geometry revealing a
remarkable relation to the mathematics of conic decomposition. Novel
geometrical measures of manifold radius and manifold dimension are introduced
which can explain the classification capacity for manifolds of various
geometries. The general theory is demonstrated on a number of representative
manifolds, including L2 ellipsoids prototypical of strictly convex manifolds,
L1 balls representing polytopes consisting of finite sample points, and
orientation manifolds which arise from neurons tuned to respond to a continuous
angle variable, such as object orientation. The effects of label sparsity on
the classification capacity of manifolds are elucidated, revealing a scaling
relation between label sparsity and manifold radius. Theoretical predictions
are corroborated by numerical simulations using recently developed algorithms
to compute maximum margin solutions for manifold dichotomies. Our theory and
its extensions provide a powerful and rich framework for applying statistical
mechanics of linear classification to data arising from neuronal responses to
object stimuli, as well as to artificial deep networks trained for object
recognition tasks.Comment: 24 pages, 12 figures, Supplementary Material
A Spectral Theory of Neural Prediction and Alignment
The representations of neural networks are often compared to those of
biological systems by performing regression between the neural network
responses and those measured from biological systems. Many different
state-of-the-art deep neural networks yield similar neural predictions, but it
remains unclear how to differentiate among models that perform equally well at
predicting neural responses. To gain insight into this, we use a recent
theoretical framework that relates the generalization error from regression to
the spectral bias of the model activations and the alignment of the neural
responses onto the learnable subspace of the model. We extend this theory to
the case of regression between model activations and neural responses, and
define geometrical properties describing the error embedding geometry. We test
a large number of deep neural networks that predict visual cortical activity
and show that there are multiple types of geometries that result in low neural
prediction error as measured via regression. The work demonstrates that
carefully decomposing representational metrics can provide interpretability of
how models are capturing neural activity and points the way towards improved
models of neural activity.Comment: First two authors contributed equally. To appear at NeurIPS 202
Linear Classification of Neural Manifolds with Correlated Variability
Understanding how the statistical and geometric properties of neural
activations relate to network performance is a key problem in theoretical
neuroscience and deep learning. In this letter, we calculate how correlations
between object representations affect the capacity, a measure of linear
separability. We show that for spherical object manifolds, introducing
correlations between centroids effectively pushes the spheres closer together,
while introducing correlations between the spheres' axes effectively shrinks
their radii, revealing a duality between neural correlations and geometry. We
then show that our results can be used to accurately estimate the capacity with
real neural data.Comment: 6 pages and 5 figures in main text. 11 pages and 1 figure in
supplementary materia
Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations
The efficient coding hypothesis proposes that the response properties of
sensory systems are adapted to the statistics of their inputs such that they
capture maximal information about the environment, subject to biological
constraints. While elegant, information theoretic properties are notoriously
difficult to measure in practical settings or to employ as objective functions
in optimization. This difficulty has necessitated that computational models
designed to test the hypothesis employ several different information metrics
ranging from approximations and lower bounds to proxy measures like
reconstruction error. Recent theoretical advances have characterized a novel
and ecologically relevant efficiency metric, the manifold capacity, which is
the number of object categories that may be represented in a linearly separable
fashion. However, calculating manifold capacity is a computationally intensive
iterative procedure that until now has precluded its use as an objective. Here
we outline the simplifying assumptions that allow manifold capacity to be
optimized directly, yielding Maximum Manifold Capacity Representations (MMCR).
The resulting method is closely related to and inspired by advances in the
field of self supervised learning (SSL), and we demonstrate that MMCRs are
competitive with state of the art results on standard SSL benchmarks. Empirical
analyses reveal differences between MMCRs and representations learned by other
SSL frameworks, and suggest a mechanism by which manifold compression gives
rise to class separability. Finally we evaluate a set of SSL methods on a suite
of neural predictivity benchmarks, and find MMCRs are higly competitive as
models of the ventral stream.Comment: Accepted at NeurIPS 202
Unsupervised learning on spontaneous retinal activity leads to efficient neural representation geometry
Prior to the onset of vision, neurons in the developing mammalian retina
spontaneously fire in correlated activity patterns known as retinal waves.
Experimental evidence suggests that retinal waves strongly influence the
emergence of sensory representations before visual experience. We aim to model
this early stage of functional development by using movies of neurally active
developing retinas as pre-training data for neural networks. Specifically, we
pre-train a ResNet-18 with an unsupervised contrastive learning objective
(SimCLR) on both simulated and experimentally-obtained movies of retinal waves,
then evaluate its performance on image classification tasks. We find that
pre-training on retinal waves significantly improves performance on tasks that
test object invariance to spatial translation, while slightly improving
performance on more complex tasks like image classification. Notably, these
performance boosts are realized on held-out natural images even though the
pre-training procedure does not include any natural image data. We then propose
a geometrical explanation for the increase in network performance, namely that
the spatiotemporal characteristics of retinal waves facilitate the formation of
separable feature representations. In particular, we demonstrate that networks
pre-trained on retinal waves are more effective at separating image manifolds
than randomly initialized networks, especially for manifolds defined by sets of
spatial translations. These findings indicate that the broad spatiotemporal
properties of retinal waves prepare networks for higher order feature
extraction